The Size of Message Set Needed for the Optimal Communication Policy

Communication is a key for facilitating multi-agent coordination on cooperative problems. In our previous work, we proposed Signal Learning (SL) and Signal Learning with Messages (SLM) by which agents learn local policies of communication and action simultaneously in MultiAgent Reinforcement Learning (MARL) framework. Our experimental results showed that both SL and SLM can improve the performance of agents’ coordination. In this paper, we focus on theoretical analysis of the conditions for constructing optimal local policies on SL and SLM framework in Decentralized Partially Observable Markov Decision Processes with Communication (Dec-POMDP-Com) models. As main results, we obtain the minimum required sizes of the message set for off-line computation of optimal local policies on SL and SLM. In addition, we report experimental results indicating that the extra messages make some positive effect in learning processes when the size of the message set is larger than the minimum required size based on theoretical analysis.